«Concurrency» in M-L-Parallel Semi-Markov Process

Chapter 8 SEMI - MARKOV PROCESS

In many (if not most) real-world applications, the arrival of customers to a service center is not well described by renewal processses. Quite often, the times between successive arrivals are correlated, whereas renewal processes have independent interarrival times. A natural generalization is the class of semiMarkov processes (SMP), which when specifically applied to the arrival of customers a...

l AVERAGE COST SEMI - MARKOV DECISION PROCESSES

^ The Semi-Markov Decision model is considered under the criterion of long-run average cost. A new criterion, which for any policy considers the limit of the expected cost Incurred during the first n transitions divided by the expected length of the first n transitions, is considered. Conditions guaranteeing that an optimal stationary (nonrandomized) policy exist are then presented. It is also ...

Hierarchical Dirichlet Process Hidden Semi-Markov Models

Given a set of sequential data in an unsupervised setting, we often aim to infer meaningful states present in the data along with characteristics that describe and distinguish those states. For example, in a speaker diarization (or who-spoke-when) problem, we are given a single audio recording of a meeting and wish to infer the number of speakers present, when they speak, and some characteristi...

The KTH Visit in Semi-Markov Processes

Parallel Algorithms for Solving Markov Decision Process

Markov decision process (MDP) provides the foundations for a number of problems, such as artificial intelligence studying, automated planning and reinforcement learning. MDP can be solved efficiently in theory. However, for large scenarios, more investigations are needed to reveal practical algorithms. Algorithms for solving MDP have a natural concurrency. In this paper, we present parallel alg...